Method and navigation system for registering two-dimensional image data set with three-dimensional image data set of body of interest

ABSTRACT

A method for registering a two-dimensional image data set with a three-dimensional image data set of a body of interest is discloses herein. The method includes the following steps: adjusting a first virtual camera according to a distance parameter calculated corresponding to the two-dimensional image data set and the body of interest; rotating the first virtual camera according to an angle difference between a first vector and a second vector; and rotating the first virtual camera according to an angle which is corresponding to a maximum similarity value of a plurality of similarity values calculated in accordance with reconstructed images of the three-dimensional image data set which includes one generated by the first virtual camera and the others generated by other virtual cameras with different angles or different pixels from the one generated by the first virtual camera and the two-dimensional image data set.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Ser.No. 63/264250, filed Nov. 18, 2021, the entire contents of which areincorporated herein by reference as if fully set forth below in itsentirety and for all applicable purposes.

BACKGROUND Field of Invention

The present disclosure relates to a method and a navigation system forestablishing a relation between a two-dimensional image data andthree-dimensional image data both corresponding to a body of interest.More particularly, the present disclosure relates to a method and anavigation system for registering a two-dimensional image data set witha three-dimensional image data set of a body of interest.

Description of Related Art

At present, the rate of occurrence of diseases corresponding to thespine is increasing day by day, and the health of human is seriouslyaffected. Spinal surgery is a main treatment for spinal diseases, andhow to minimize wounds during and after a procedure becomes more andmore important, such that infection risks can be reduced and patientsare able to quickly recover without hospitalization. For this purpose,an image guided surgical procedure with a navigation system isintroduced into spinal surgeries. During the image guided medicalprocedure, the area of interest of a patient that has been imaged isdisplayed on a display of the navigation system. Meanwhile, the systemcan track surgical instruments and/or implants and then integrate theirsimulated images with the area of interest of the patient body. Bytaking advantage of such a procedure and system, physicians are able tosee the location of the instruments and/or implants relative to a targetanatomical structure without the need to frequent use of C-Armfluoroscopy throughout the entire surgical procedure is generallydisclosed in U.S. Pat. No. 6,470,207, entitled “Navigational GuidanceVia Computer-Assisted Fluoroscopic Imaging,” issued Oct. 22, 2002, whichis incorporated herein by reference in its entirety.

However, the spine navigation system usually needs to use the X-rayinspection for obtaining the internal body of the patient. The patientis therefore in danger due to the radiation exposure. In view of theforegoing, problems and disadvantages are associated with existingsystems that require further improvement. However, those skilled in theart have yet to find a solution.

There are many ways to achieve two-directional (2D) to three-directional(3D) registration by adopting obtained 2D images to register 3D imageswhich are known in the art. The registrations from 2D to 3D includecontour algorithms, point registration algorithms, surface registrationalgorithms, density comparison algorithms, and pattern intensityregistration algorithms. The foregoing registrations, however, need agreat deal of calculation and therefore, usually take several minutes toexecute the 2D to 3D registration, Actually, some of these registrationsmay take upwards of twenty minutes to an hour to execute theregistration. Furthermore. these registration processes could alsoresult in an inexact registration after waiting for such long time.

In view of the foregoing, problems and disadvantages are associated withexisting technologies, that require further improvement, and there is aneed to provide a method and equipment for executing 2D to 3Dregistration in a more accurate and efficient way. Moreover, the flowsof the method for executing 2D to 3D registration shall also beimproved.

In addition, 2D images (e.g., X-ray images) generally cover more thanone level of vertebrae. Matching 2D images with digitally reconstructedradiographs (DRR) from computed tomography (CT) data is not efficientand easy to cause registration failures, errors, or inaccuracy due tointerference in regions of interest such as pedicle screw, cage, ribs,and Ilium. Specialized algorithm focusing on separated local comparisoninstead of global matching may be used to help the comparison in suchanatomical images. Even if a 2D to 3D registration is performed, thereare still problems and disadvantages, for example, more time consumptionand less accuracy. Therefore, how to make the 2D to 3D registrationbecome faster and more accurate is a purpose for the industry. Inparticular, considering separated local comparison is introduced into 2Dto 3D registration, developing a pre-process capable to further shortenthe registration period and improve the accuracy is in demand.

SUMMARY

The foregoing presents a simplified summary of the disclosure in orderto provide a basic understanding to the reader. This summary is not anextensive overview of the disclosure and it does not identifykey/critical elements of the present disclosure or delineate the scopeof the present disclosure. Its sole purpose is to present some conceptsdisclosed herein in a simplified form as a prelude to the more detaileddescription that is presented later.

One aspect of the present disclosure is to provide a method forregistering a two-dimensional image data set with a three-dimensionalimage data set of a body of interest. The method comprises followingsteps: adjusting a first virtual camera according to a distanceparameter calculated corresponding to the two-dimensional image data setand the body of interest: rotating the first virtual camera according toan angle difference between a first vector and a second vector, whereinthe first vector is calculated from two spatial marks in thethree-dimensional image data set, and the second vector is calculatedfrom two first plain marks in the two-dimensional image data set: androtating the first virtual camera according to an angle which iscorresponding to a maximum similarity value of a plurality of similarityvalues calculated in accordance with reconstructed images of thethree-dimensional image data set which includes one generated by thefirst virtual camera and the others generated by other virtual cameraswith different angles or different pixels from the one generated by thefirst virtual camera and the two-dimensional image data set forimplementing in the two-dimensional image data set to thethree-dimensional image data set registration of a navigation systemafter adjusting and rotating.

Another aspect of the present disclosure is to provide a navigationsystem for registering a two-dimensional image data set with athree-dimensional image data set of a body of interest. The navigationsystem comprises a memory and a processor. The memory is configured tostore a plurality of commands. The processor is configured to obtain theplurality of commands from the memory to perform following steps:adjusting a first virtual camera according to a distance parametercalculated corresponding to the two-dimensional image data set and thebody of interest; rotating the first virtual camera according to anangle difference between a first vector and a second vector, wherein thefirst vector is calculated from two spatial marks in thethree-dimensional image data set, and the second vector is calculatedfrom two first plain marks in the two-dimensional image data set; androtating the first virtual camera according to an angle which iscorresponding to a maximum similarity value of a plurality of similarityvalues calculated in accordance with reconstructed images of thethree-dimensional image data set which includes one generated by thefirst virtual camera and the others generated by other virtual cameraswith different angles or different pixels from the one generated by thefirst virtual camera and the two-dimensional image data set forimplementing in the two-dimensional image data set to thethree-dimensional image data set registration of the navigation systemafter adjusting and rotating.

In view of the foregoing, the method and the navigation system of thepresent disclosure may provide an initialization alignment before atypical 2D to 3D registration process. Since an additional alignmentsuch as the initialization alignment is performed in advance, theoverall 2D to 3D registration becomes faster and more accurate. It is tobe understood that both the foregoing general description and thefollowing detailed description are by examples, and are intended toprovide further explanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide a furtherunderstanding of the invention, and are incorporated in and constitute apart of this specification. The drawings illustrate embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention. In the drawings.

FIG. 1 depicts a schematic diagram of a navigation system according toone embodiment of the present disclosure;

FIG. 2 depicts a schematic diagram of a calculating device, a database,and an optical tracker of the navigation system shown in FIG. 1according to one embodiment of the present disclosure;

FIG. 3 depicts a flow diagram of a method for registering atwo-dimensional image data set with a three-dimensional image data setof a body of interest according to one embodiment of the presentdisclosure;

FIG. 4 depicts a schematic diagram of a portion of a body of interestaccording to one embodiment of the present disclosure;

FIG. 6 depicts a schematic diagram of a portion of a body of interestaccording to one embodiment of the present disclosure;

FIG. 6 depicts a schematic diagram of a portion of a body of interestaccording to one embodiment of the present disclosure;

FIG. 7 depicts a schematic diagram of a portion of a body of interestaccording to one embodiment of the present disclosure;

FIG. 8 depicts a schematic diagram of operations of the C-arm deviceshown in FIG. 1 according to one embodiment of the present disclosure,

FIG. 9 depicts a schematic diagram of operations of the C-arm deviceshown in FIG. 1 according to one embodiment of the present disclosure;

FIG. 10 depicts a flow diagram of a method for registering atwo-dimensional image data set with a three-dimensional image data setof a body of interest according to one embodiment of the presentdisclosure;

FIG. 11 depicts a flow diagram of a method for registering atwo-dimensional image data set with a three-dimensional image data setof a body of interest according to one embodiment of the presentdisclosure;

FIG. 12 depicts a flow diagram of a method for registeringtwo-dimensional image data set with a three-dimensional image data setof a body of interest according to one embodiment of the presentdisclosure;

FIG. 13 depicts a flow diagram of a method for registering atwo-dimensional image data set with a three-dimensional image data setof a body of interest according to one embodiment of the presentdisclosure;

FIG. 14 depicts a schematic diagram of a virtual camera simulated by thecalculating device of the navigation system shown in FIG. 1 according toone embodiment of the present disclosure;

FIG. 15 depicts a flow diagram of a method for registering atwo-dimensional image data set with a three-dimensional image data setof a body of interest according to one embodiment of the presentdisclosure; and

FIG. 16 depicts a flow diagram of a method for registering atwo-dimensional image data set with a three-dimensional image data setof body of interest according to one embodiment of the presentdisclosure.

According to the usual mode of operation, various features and elementsin the figures have not been drawn to scale, which are drawn to the bestway to present specific features and elements related to the disclosure.In addition, among the different figures, the same or similar elementsymbols refer to similar elements/components.

DESCRIPTION OF THE EMBODIMENTS

To make the contents of the present disclosure more thorough andcomplete, the following illustrative description is given with regard tothe implementation aspects and embodiments of the present disclosure,which is not intended to limit the scope of the present disclosure. Thefeatures of the embodiments and the steps of the method and theirsequences that constitute and implement the embodiments are described.However, other embodiments may be used to achieve the same or equivalentfunctions and step sequences.

Unless otherwise defined herein. scientific and technical terminologiesemployed in the present disclosure shall have the meanings that arecommonly understood and used by one of ordinary skill in the art. Unlessotherwise required by context, it will be understood that singular termsshall include plural forms of the same and plural terms shall includethe singular. Specifically, as used herein and in the claims, thesingular forms “a” and “an” include the plural reference unless thecontext clearly indicates otherwise.

FIG. 1 depicts a schematic diagram of a navigation system 1000 having acalculating device 1100, a database 1200, and an optical tracker 1400according to one embodiment of the present disclosure. The navigationsystem 1000 can be used to navigate an instrument 9500 during a surgicaloperation of an object 9000, but the present disclosure is not limitedthereto. In some embodiments, the navigation system 1000 can be used tonavigate any kind of tools during any type of operations depending onactual requirements. For instances, the navigation system 1000 can beused to navigate screws, graspers, clamps, surgical scissors, cutters,needle drivers, retractors, distracters, dilators, suction tips andtubes, sealing devices, irrigation and injection needles, powereddevices, scopes and probes, carriers and appliers, ultrasound tissuedisruptors, cryotomes and cutting laser guides, and measurement devices.The navigation system 1000 can also be used to navigate any type ofimplant including orthopedic implants, spinal implants, cardiovascularimplants, neurovascular implants, soft tissue implants, or any otherdevices implanted in an objective 9000 such as a patient.

The calculating device 1100 can be but not limited to a computer, adesktop, a notebook, a tablet, or any other devices that can performcalculating function. The database 1200 can be but not limited to a datastorage, a computer, a server, a cloud storage, or any other devicesthat can store data. The database 1200 is used to store, pre-acquiredthree-dimensional (3D) image data set of the object 9000. The 30 imagedata set can be obtained by using a computerized tomography (CT) to scanthe object 9000.

The imaging device 1300 can be established and performed separately fromthe navigation system 1000, and the imaging device 1300 can be but notlimited to a C-arm mobile fluoroscopy machine or a mobile X-ray imageintensifier The imaging device 1300 includes a X-ray emitter 1310 and aX-ray receiver 1320. The former emits X-rays that penetrate the body ofthe object 9000, and the latter receives and converts the X-rays intodigital two-dimensional (2D) images on a display of imaging device 1300.According to the planes along which the imaging device 1300 obtainsimages, the 2D images may be an anterior/posterior (AP) image and alateral view (LA) 2D image of the object 9000. The object 9000 can bebut not limited to a patient.

51 The calibrator 1510 including calibration markers is used to bothcalibrate the imaging device 1300 and track the location of the imagingdevice 1300 during an image is obtained. The calibrator 1510 is includedin or disposed on the X-ray receiver 1320, and the dynamic referenceframes 1520A are disposed on or close to the X-ray receiver 1320 and thebody of interest of the object 9000. The calibrator 1510 is in the pathfrom the X-ray emitter 1310 to the X-ray receiver 1320 and opaque-orsemi-opaque to the X-ray. Therefore, the X-ray entering the object 9000and also the calibrator 1510 are partially absorbed by particulartissues and the calibration markers in the calibrator 1510 beforereceived by the X-ray receiver 1320 in operation. This makes thetwo-dimensional images present the body of interest of the object 9000and the calibration markers of the calibrator 1510. The calculatingdevice 1100 may acquire the relation of the X-ray emitter 1310 and theX-ray receiver 1320 by calculating the pattern of the calibrationmarkers presented on the two-dimensional images. The AP two-dimensionalimage and the LA two-dimensional image of the vertebral bodies ofinterest of the object 9000 are involved the two-dimensional image dataset in the present embodiment. Alternatively, in another embodiment, itcan be only AP two-dimensional image or LA two-dimensional image in thetwo-dimensional image data set. Then, the two-dimensional image data setcan be transmitted as an electronic file to the calculating device 1100for later use.

The three-dimensional image data set in the database 1200, thetwo-dimensional image data set generated from the imaging device 1300,the optical tracker 1400, the calibrators 1510, the dynamic referenceframe 1520A, and the object 9000 have their own coordinates in differentcoordinate systems. The navigation system 1000 of the present disclosurecan establish relations among those coordinate systems, such thatsurgeons may use the navigation system 1000 to navigate the instrument9500 or during the surgical operation. The relation among thosecoordinates is described below. The calibrator 1510 and the dynamicreference frame1520A are disposed on the X-ray receiver 1320, thelocation of the three can be substantially treated as the same, andtherefore, the relation between the calibrator coordinate system and theimaging device coordinate system is established. The optical tracker1400 then tracks the dynamic reference frame 1520A including reflectorto introduce the calibrator 1510, the dynamic reference frame 1520A, andthe X-ray receiver 1320 into the tracker coordinate system, andtherefore, the location of the X-ray receiver 1320 in the trackercoordinate system is obtained. In some embodiments, the optical tracker1400 can o track the calibrator 1530 including reflector to introducethe calibrator 1530 into the tracker coordinate system, and therefore,the location of the calibrator 1530 in the tracker coordinate system isobtained. Since the relation of the X-ray emitter 1310 and the X-rayreceiver 1320 is acquired as described above, the location of the X-rayemitter 1310 in the tracker coordinate system can be acquired as well.It is noted that, the present disclosure is not limited to thestructures and the operations as shown in FIG. 1 , and it is merely anexample for illustrating one of the implements of the presentdisclosure.

It is noted that the registration procedure of the navigation system1000 substantially consists of three stages which are initializationalignment, 2D to 3D registration, and realignment. In some embodiments,the program installed in the navigation system 1000 runs through thesethree stages, further discussed herein. Alternatively, in otherembodiments, the program can run only the initial alignment process, the2D to 3D registration, the re-registration, or a combination thereofdepending on the situations and/or the functions needed. More detailswill be discussed below.

Initialization Alignment

Before using the navigation system 1000, an initial alignment processwould be performed to the navigation system 1000 so as to improve theefficiency and enhance the precision of the navigation system 1000,which will be described below.

FIG. 2 depicts a schematic diagram of the calculating device 1100, thedatabase 1200, and the optical tracker 1400 of the navigation system1000 shown in FIG. 1 according to one embodiment of the presentdisclosure. As shown in the figure, the calculating device 1100 iselectrically connected to the database 1200 and the imaging device 1300.

The calculating device 1100 includes a memory 1110, a processor 1120,and an I/O interface 1130. The processor 1120 is electrically connectedto the memory 1110 and the I/O interface 1130. The memory 1110 is usedto store a plurality of commands, and the processor 1120 is used toobtain the plurality of commands from the memory 1110 to perform stepsof a method 2000 shown in FIG. 3 for registering a two-dimensional imagedata set with a three-dimensional image data set of a body of interest.

In some embodiments, the memory 1110 can include, but not limited to, atleast one of a flash memory, a hard disk drive (HDD), a solid-statedrive (SSD), a dynamic random access memory (DRAM) and a static randomaccess memory or a combination thereof. In some embodiments, as being anon-transitory computer readable medium, the memory 1110 can store, thecomputer readable commands that can be accessed by the processor 1120.

In some embodiments, the processor 1120 can include, but not limited to,a single processor or an integration of multiple microprocessors, suchas a central processing unit (CPU), a graphics processing unit (GPU) oran application-specific integrated circuit (ASIC), etc.

Reference is now made to both FIG. 2 and FIG. 3 . In operation, the step2100 is performed by the processor 1120 to adjust a first virtual cameraaccording to a distance parameter calculated corresponding to thetwo-dimensional image data set and the body of interest.

For example, the processor 1120 of the calculating device 1100 mayobtain the X-ray images directly from the imaging device 1300 orpre-stored in the database 1200. As discussed above, the two-dimensionaldata set includes at least the AP and LA two-dimensional images whichboth include the calibrator 1510 in FIG. 1 . The calibrator 1510 can beused to identify the location of the imaging device 1300 in operation inaccordance with the abovementioned calculation of the relation. Once thelocation of the imaging device 1300 is acquired, the processor 1120 mayestimate the distance between the X-ray emitter 1310 and the targetedvertebral body of interest in the objective 9000. The estimated distancethen is converted to a distance parameter for by the processor 1120 ofthe calculating device 1100.

The initialization alignment 2000 according to the embodiment disclosedherein is illustrated. Briefly, the initialization alignment 2000 caninclude three steps 2100, 2200, 2300.

As a first step, the surgeon is asked to acquire a first image or ananterior/posterior (AP) image and a second image or a lateral (LA) imagefrom the imaging device 1300. These images are used for refinement andnavigation, but also to initialize the orientation. Since the imagingdevice 1300 includes the calibrator 1510 the location of the X-rayemitter 1310 during image acquisition with respect to the patientcoordinate system is known (i.e., patient AP direction and patient LAdirection). This knowledge is combined with the knowledge of how thepatient 9000 is oriented during the three-dimensional volume scan by thesetting of the user. Given the estimated orientation, digitallyreconstructed radiographs (DRRs) are created from the three-dimensionaldata set acquired by CT scan. Owing to the DRRs being a kind of digitalimages, it is helpful to explain how and where they are generated withvirtual cameras. That is to say, the AP and LA DRRs are captured by twovirtual cameras simulatively for example located at the particular spotstoward to the vertebral body of interest of the patient 9000 along theAP and LA planes respectively. The DRRs correspond to the actualinteroperative AP and lateral LA radiographs. The surgeon is presentedwith the DRRs and the actual radiographs and is asked to identify acommon point in all images. This step provides a common point orposition in the CT image data and the dynamic reference frame 1520A orpatient space coordinates. Once the position and orientation of thepatient 9000 in the three-dimensional data set and the dynamic referenceframe coordinates are known, the method proceeds to the refinement step.After refinement, all of the systems involved are linked and known.

During the initialization step, the surgeon is asked to acquire AP andLA radiographs or radiographs along any two planes using the imagingdevice 1300. In a spinal procedure, the surgeon is prompted to identifythe center of the vertebral body that the surgeon is interested inputting together all of this data, a good estimate of the orientationand the position is known and is used to calculate the DRRs thatcorrespond closely to the actual radiographs for an initialtwo-dimensional to three-dimensional registration. These DRRs arecreated by adopting the three-dimensional data from the CT scan combinedwith the information from the C-arm localization target.

The processor 1120 of the calculating device 1100 may access thethree-dimensional image data set stored in the database 1200, and an APvirtual camera or an LA virtual camera corresponding to thethree-dimensional image data set can be created by the processor 1120.Each of the virtual cameras has its own spatial parameters indicatingthe position and orientation corresponding to the body of interest inthe three-dimensional image. Just like an actual camera, that thepositions and orientations of the cameras corresponding to the targetobjective determines what images will be obtained. Therefore, changingthe spatial parameter of a virtual camera will generate differentreconstructed images from the three-dimensional image data set. In thepresent embodiment, the spatial parameter includes at least a distanceparameter which indicates the distance from the target body of interestof the object 9000 to the virtual camera.

Preliminary Registration

In the preliminary registration step, software is used to providematching algorithms that reprocess the preliminary two-dimensional tothree-dimensional registration. The software is adopting to adjust theinitial position and the orientation in a way as to minimize differencesbetween the DRRs and the actual radiographs, thereby refining theregistration. In the preliminary step, similarity or cost measures areimplemented to identify how well the images match. An iterativepreliminary algorithm adjusts the initial position and the orientationparameters simultaneously to maximize the similarity between the DRRsand the actual radiographs. The similarity or cost measures that areimplemented are selected from known similarity and cost measures, suchas normalized mutual information, mutual information, gradientdifference algorithms, surface contour algorithms, pattern intensityalgorithms, sum of squared differences, normalized cross-correlation,local normalized correlation, and correlation ratio. This procedureresults in an efficient and accurate manner to provide two-dimensionalto three-dimensional registration.

Subsequently, the processor 1120 of the calculating device 1100 mayadjust the AP virtual camera or the LA virtual camera according to thedistance parameter.

In some embodiments, the distance parameter is calculated according tothe actual distance between two plain marks on an image of thetwo-dimensional image data set. For example, surgeons may be asked tomanually make two marks on the single AP or LA two-dimensional imagewhere they are interested in.

In the present embodiment shown in FIG. 4 , the object 9000 is a patientunder a vertebral disorder, the surgeon may manually make two marks V0and V1 on but not limited to the center of two target vertebral bodiesof the object 9000 through the I/O interface 1130 in FIG. 2 . As such,the distance between the marks V0 and V1 can be calculated by theprocessor 1120 of the calculating device 1100 with an imagingrecognition technology. In some embodiments, the I/O interface 1130 canbe electrically connected to any kind of input devices such as a mouse,a keyboard, and a touch panel for inputting.

As can be seen in FIG. 5 , it is an AP digitally reconstructedradiograph (DRR) image generated by the processor 1120 with inputscombining the information of where the object 9000 was positioned ororiented corresponding to the imaging device 1300 during X-ray imageacquisition.

Virtual camera in some embodiments is a modularized coding of thetwo-dimensional to three-dimensional registration program for generationof two-dimensional image from a three-dimensional volume at aparameter-driven position along with a parameter-driven orientation orperspective by the processor 1120 with known DRR algorithm.

Using this estimate of the orientation, the DRR image in FIG. 5 from theCT scan is created to correspond substantially to the actualinteroperative radiograph in FIG. 4 . By using the patient orientationinformation from the three-dimensional image data with the patientorientation information on the location of the imaging device 1300, thisinformation is implemented in combination with known DRR algorithms tocreate the DRR. In other words, since it is known where the fluoroscopicscans were taken, and the right/left and AP directions of the CT data orany other directions are known, if a person takes a view through thethree-dimensional volume data along the direction of the fluoroscopicscan and an accurate DRR is obtained. The DRR is essentially atwo-dimensional image taken from a three-dimensional volume where a viewthrough the three-dimensional volume is generated by looking throughthree-dimensional volume from the proper orientation or perspective, tocreate a line for every voxel. Each line is considered to create a newvoxel value, which generates the DRR.

However, the initial DRRs are not precisely matching with the AP and LAX-ray images. That is why there is a need to perform an alignmentprocedure during two-dimensional to three-dimensional registration.

The surgeons may manually make another two marks V0′ and V1′ on thetarget two vertebral bodies of the AP DRR image through the I/Ointerface 1130 in FIG. 2 . As such, a distance between two marks V0′ andV1′ on the AP DRR image in FIG. 5 can be calculated by the processor1120. Subsequently, if the distance between two marks V0 and V1 on theAP 2D image in FIG. 4 is, for example, 100 pixels. the processor 1120may adjust the distance parameter of the AP virtual camera to regeneratean adjusted AP DRR image which has the distance between two marks V0′and V1′ to be also 100 pixels. Therefore, the size of the targetvertebral bodies shown on the AP DRR image in FIG. 5 is adjusted, to besimilar to the size of those shown on the AP 2D image in FIG. 4 . Inview of the above, it merely needs to focus on the distances regardingthe marks on the AP 2D image in FIG. 4 and the AP DRR image in FIG. 5 ,and then adjusts the distances to be the same for alignment, andtherefore, the alignment process can be very fast.

Besides, in the present embodiment shown in FIG. 6 , the surgeons maymanually make marks V0 and V1 on two target vertebral bodies of theobject 9000 through the I/O interface 1130 in FIG. 2 . As such, thedistance between the marks V0 and V1 can be calculated by the processor1120 of the calculating device 1100 with an imaging recognitiontechnology.

As can be seen in FIG. 7 , it is a LA DRR image generated by theprocessor 1120 with inputs combining the information of where the object9000 was positioned or oriented corresponding to the imaging device 1300during X-ray image acquisition. The surgeon may manually make two markV0′ and V1′ on two target vertebral bodies of the object through the I/Ointerface 1130 in FIG. 2 . As such, the distance between the marks V0′and V1′ can be calculated by the processor 1120 of the calculatingdevice 1100 with an imaging recognition technology. Subsequently, if thedistance between two marks V0 arid V1 on the LA 2D image in FIG. 6 is,for example, 100 pixels, the processor 1120 may adjust the distanceparameter of the LA virtual camera to regenerate an adjusted LA DRRimage which has the distance between two marks V0′ and V1′ to be also100 pixels. Therefore, the size of the target vertebral bodies shown onFIG. 7 is adjusted to be similar to the size of those shown on the LA 2Dimage in FIG. 6 . In view of the above, it merely needs to focus on thedistances regarding the marks on the LA 2D image in FIG. 6 and the LADRR image in FIG. 7 , and then adjusts the distances to be the same foralignment, and therefore, the, alignment process can be very fast.

In some embodiments the distance parameter is calculated according to adistance between an estimated position of an emitter and an estimatedposition of the body of interest. For example, as discussed above, thecalibrator 1510 shown on the X-ray two-dimensional image generated bythe imaging device 1300 in FIG. 1 can be used as a reference for thecalculating device 1100 to calculate the estimated position of the X-rayemitter 1310. In addition, the estimated position of the object 9000 canbe calculated by the calculating device 1100 according to the X-rayimage showing the calibrators 1510 and the estimated position of theX-ray emitter 1310. Thereafter, if the estimated position of the X-rayemitter 1310 and the estimated position of the object 9000 are obtained,the distance between the X-ray emitter 1310 and the estimated positionof the object 9000 is able to be calculated by the calculating device1100.

In some embodiments, the estimated position of the emitter is calculatedaccording to the two-dimensional image data set, and the estimatedposition of the body of interest is at a position at which a firstvirtual line and a second virtual line are the closest. For example, thecalculating device 1100 in FIG. 2 may obtain the X-ray images from theimaging device 1300. As discussed above, the X-ray images from theimaging device 1300 may include the calibrator 1510 in FIG. 1 .Therefore, the calculating device 1100 may calculate the estimatedposition of the X-ray emitter 1310 according to the X-ray imageincluding the calibrator 1510. Referring to FIG. 8 , there is a firstvirtual line VL1 which is generated between the estimated position ofthe X-ray emitter 1310 and the calibrator 1510. Referring to FIG. 9 ,there is a second virtual line VL2 which is generated between theestimated position of the X-ray emitter 1310 and the calibrator 1510. Ifthe first virtual line VL1 in FIG. 8 and the second virtual line VL2 inFIG. 9 are located in the same coordinate system, the estimated positionof the object 9000 is determined at a position at which the firstvirtual line VL1 and the second virtual line VL2 are the closest.

In some embodiments, the two-dimensional image data set includes firstand second two-dimensional images, the first virtual line is generatedbetween the estimated position of the emitter and a central point of atleast two reflectors which are radiated when the first two-dimensionalimage data set is captured, the second virtual line is generated betweenthe estimated position of the emitter and the central point of the atleast two reflectors which are radiated when the second two-dimensionalimage data set is captured.

For example, the X-ray images from the imaging device 1300 in FIG. 1include the AP 2D image and the LA 2D image of the object 9000.Referring to FIG. 8 , the first virtual line VL1 is generated betweenthe estimated position of the X-ray emitter 1310 and a central point1521 of reflectors 1523, 1525, 1527 which can be related to the dynamicreference frame 1520A in FIG. 1 and are obtained by the optical tracker1400. Referring to FIG. 9 , the second virtual line VL2 is generatedbetween the estimated position of the X-ray emitter 1310 and a centralpoint 1521 of reflectors 1523, 1525, 1527 which can be related to thedynamic reference frame 1520A in FIG. 1 and obtained by the opticaltracker 1400. It is noted that the central point 1521 can be an originalpoint of the calibrator coordinate constructed by the reflectors 1523,1525, 1527, and the location of the central point 1521 in not limited tothe physical center of the reflectors 1523, 1525, 1527.

Referring to both FIG. 2 and FIG. 3 , in operation, the step 2200 isperformed by the processor 1120 to rotate the first virtual cameraaccording to an angle difference between a first vector and a secondvector, wherein the first vector is calculated from two spatial marks inthe three-dimensional image data set, and the second vector iscalculated from two first plain marks in the two-dimensional image dataset.

For example, the first vector is calculated from two spatial marks inthe three-dimensional image generated by the AP virtual camera withproper DDR algorithm. Referring to FIG. 4 , the second vector iscalculated from two plain marks V0 and V1 on the AP two-dimensionalimage. When the first vector and the second vector are obtained, theprocessor 1120 may calculate an angle difference between the firstvector and the second vector. Subsequently, the processor 1120 canrotate the AP virtual camera according to the angle difference.Therefore, the orientation or perspective of the AP virtual camera isadjusted to be similar or identical in the best to that of the X-rayemitter 1310 of the imaging device 1300 along the AP plain in FIG. 4 .This step is executed by the calculating device 1100 to adjust or modifythe orientation or perspective parameter of the DRR algorithm so as togenerate an adjusted or modified DRR image which is more similar oridentical in the best to the AP two-dimensional X-ray image. In view ofthe above, the vertebral bodies in the AP two-dimensional image and theAP DRR image can be adjusted to be identical, and the elevation anglesregarding the LA two-dimensional image and the LA DRR image can beadjusted to be similar.

Furthermore, the first vector is calculated from two spatial marks inthe three-dimensional image generated by the LA virtual camera withproper DDR algorithm. Referring to FIG. 6 , the second vector iscalculated from two plain marks V0 and V1 on the LA two-dimensionalimage. When the first vector and the second vector are obtained, theprocessor 1120 may calculate an angle difference between the firstvector and the second vector. Subsequently, the processor 1120 canrotate the LA virtual camera according to the angle difference.Therefore, the orientation or perspective of the LA virtual camera isadjusted to be similar or identical in the best to that of the X-rayemitter 1310 of the imaging device 1300 along the LA plain in FIG. 6 .This step is executed by the calculating device 1100 to adjust or modifythe orientation or perspective parameter of the DRR algorithm so as togenerate an adjusted or modified DRR image which is more similar oridentical in the best to the LA two-dimensional X-ray image. In view ofthe above, the vertebral bodies in the LA two-dimensional image and theLA DRR image can be adjusted to be identical, and the elevation anglesregarding the AP two-dimensional image and the AP DRR image can beadjusted to be similar.

In some embodiments, the two-dimensional image data set includes a firsttwo-dimensional image, the first vector is calculated from the twospatial marks in the three-dimensional image data set generated by thefirst virtual camera, and the second vector is calculated by the twofirst plain marks in the first two-dimensional image data For example,the two-dimensional image data set includes the AP two-dimensional imagein FIG. 4 . The first vector is calculated from two spatial marks in thethree-dimensional image data set generated by the AP virtual camera. Thesecond vector is calculated by two plain marks V0 and V1 in the APtwo-dimensional image in FIG. 4 .

In some embodiments, the two-dimensional image data set includes asecond two-dimensional image, and the second vector is calculated fromthe two second plain marks in the second two-dimensional image data. Forexample, the two-dimensional image data set includes the LAtwo-dimensional image in FIG. 6 . The first vector is calculated fromtwo spatial marks in the three-dimensional image data set generated bythe LA virtual camera. The second vector is calculated from the twoplain marks V0 and V1 in the LA two-dimensional image in FIG. 6 .

Reference is now made to both FIG. 2 and FIG. 3 . In operation, the step2300 is performed by the processor 1120 to rotate the first virtualcamera according to an angle which is corresponding to a maximumsimilarity value of a plurality of similarity values calculated inaccordance with reconstructed images of the three-dimensional image dataset which includes one generated by the first virtual camera and theothers generated by other virtual cameras with different angles ordifferent pixels from the one generated by the first virtual camera andthe two-dimensional image data set for implementing in thetwo-dimensional image data set to the three-dimensional image data setregistration of the navigation system 1000 after adjusting and rotating.

For example, first, a few LA DRR images are generated by adjusting thepre-determined position and/or orientation of the LA virtual camera inthe coordinate system of the three-dimensional image data set. Thepre-determined position and/or orientation of the LA virtual camera canbe chosen by the surgeon in accordance with the volume of interest inthe three-dimensional data set.

The LA DRR images can be generated by taking the pre-determined positionand/or orientation of the LA virtual camera as the center and thenrotating its roll angle in a range from −20 degrees to 20 degrees and/ormoving its position in a range from −15 pixels to 15 pixels for DRRgeneration. Since different LA DRR images have different contents, aplurality of similarity values can be calculated in accordance with theLA DRR images and the pre-determined LA two-dimensional image, which ischosen by the surgeon in accordance with the region of interestcorresponding to the volume of interest. Then, the maximum similarityvalue will be obtained from the plurality of similarity values. For ahigher alignment between the two-dimensional and DRR image, the currentposition and/or orientation of the LA virtual camera will be adjustedaccording to the pixel and roll angle corresponding to the maximumsimilarity value. In other words, the vertebras in the LA DRR image areadjusted to be similar to the vertebras of the LA two-dimensional image.The AP virtual camera regarding each vertebral bodies of the object 9000can be adjusted according to LA virtual camera in accordance withcorresponding vertebral bodies of the object 9000.

In addition, a few AP DRR images are generated by adjusting thepre-determined position and/or orientation of the AP virtual camera inthe coordinate system of the three-dimensional image data set. Thepre-determined position and/or orientation of the AP virtual camera canbe chosen by the surgeon in accordance with the volume of interest inthe three-dimensional data set.

The AP DRR images can be generated by taking the pre-determined positionand/or orientation of the AP virtual camera as the center and thenrotating its azimuth angles in a range from −15 degrees to 15 degreesand/or moving its position in a range from −15 pixels to 15 pixels forDRR generation. Since different AP DRR images have different contents, aplurality of similarity values can be calculated in accordance with theAP DRR images and the pre-determined AP two-dimensional image, which ischosen by the surgeon in accordance with the region of interestcorresponding to the volume of interest. Then, the maximum similarityvalue will be obtained from the plurality of similarity values. For ahigher alignment between the two-dimensional and DRR image, the currentposition and/or orientation, of the AP virtual camera will be adjustedaccording to the pixel and azimuth angle corresponding to the maximumsimilarity value. In other words, the vertebras in the AP DRR image areadjusted to be similar to the vertebras of the AP 2D image. The LAvirtual camera regarding each vertebral bodies of the object 9000 can beadjusted according to AP virtual camera in accordance with correspondingvertebral bodies of the object 9000.

In some embodiments, an adjusted first virtual camera is generated afterthe rotation of the first virtual camera. The processor 1120 is used toperform a step of rotating the adjusted first virtual camera accordingto the angle which is corresponding to an adjusted maximum similarityvalue of the plurality of similarity values calculated in accordancewith adjusted reconstructed images which includes one generated by theadjusted first virtual camera and the others generated by other virtualcameras with different angles from the one generated by the adjustedfirst virtual camera and the two-dimensional image data set.

For example, adjusted AP virtual camera or adjusted LA virtual cameraare generated after the rotation of the AP virtual camera or the LAvirtual camera. The adjusted LA DRR images are generated by the adjustedLA virtual camera with roll angles in a range from −20 degrees to 20degrees and/or moving its position in a range from −15 pixels to 15pixels far DRR generation. Since different LA DRR images have differentcontents, a he plurality of similarity values can be calculated inaccordance with the adjusted LA DRR images and the pre-determined LAtwo-dimensional imager which is chosen by the surgeon in accordance withthe region of interest corresponding to the volume of interest. Then,the adjusted maximum similarity value will be obtained from theplurality of similarity values. For a higher alignment between thetwo-dimensional and DRR image, the current position and/or orientationof the LA virtual camera will he adjusted according to the pixel androll angle corresponding to the maximum similarity value. In otherwords, the vertebras in the LA DRR image are adjusted to be similar tothe vertebras of the LA two-dimensional image.

Besides, the adjusted AP DRR images are generated by the adjusted APvirtual camera with azimuth angles in a range from −15 degrees to 15degrees and/or moving its position in a range from −15 pixels to 15pixels for DRR generation. Since different APDRR images have differentcontents, a be plurality of similarity values can be calculated inaccordance with the adjusted AP DRR images and the AP two-dimensionalimage, which is chosen by the surgeon in accordance with the region ofinterest corresponding to the volume of interest. Then, the adjustedmaximum similarity value will be obtained from the plurality ofsimilarity values. For a higher alignment between the two-dimensionaland DRR image, the current position and/or orientation of the AP virtualcamera will be adjusted according to the pixel and roll anglecorresponding to the maximum similarity value. In other words, thevertebras in the AP DRR image are adjusted to be similar to thevertebras of the AP two-dimensional image.

It is noted that. the present disclosure is not limited to thestructures and the operations as shown in FIG. 2 to FIG. 9 , and it ismerely an example for illustrating one of the implements of the presentdisclosure.

In some embodiments, the processor 1120 is used to perform a step ofadjusting the first virtual camera of the plurality of virtual camerascorresponding to the three-dimensional image data set according to amatrix corresponding to the two-dimensional image data set. Far example,both of the AP two-dimensional image and the LA two-dimensional imagefrom the imaging device 1300 in FIG. 1 includes the calibrator 1510. Theprocessor 1120 may calculate the matrix which functions to transform thecoordinate system of the AP two-dimensional image to the coordinatesystem of the LA two-dimensional image by the given parameter of thecalibrator 1510. Therefore, the matrix corresponding to the relationbetween the AP two-dimensional image and the LA two-dimensional image isobtained. Subsequently, the processor 1120 may adjust the AP virtualcamera or the LA virtual camera according to the matrix, such that therelation between the AP virtual camera and the LA virtual camera issimilar to the relation between the AP two-dimensional image and the LAtwo-dimensional image.

2D to 3D Registration

Once the initialization alignment as the first stage is completed, thenavigation system 1000 proceeds to establish the relation between thethree-dimensional image data set stored in the database 1200 and theX-ray two-dimensional images of the two-dimensional image data set shownon the monitor of the imaging device 1300 or stored in the database1200, which will be described below.

FIG. 10 depicts a flow diagram of a method for registering at least onetwo-dimensional image of the two-dimensional image data set of a body ofinterest with a three-dimensional image data set of the same body ofinterest according to one embodiment of the present disclosure.Reference is now made to both FIG. 2 and FIG. 10 . In operation, thestep 3100 is performed by the processor 1120 to generate a firstreconstructed image from the three-dimensional image data set with afirst spatial parameter.

For example, the processor 1120 of the calculating device 1100 mayaccess the database 1200 and simulate a virtual camera corresponding tothe three-dimensional image data set with the first spatial parameter soas to generate a first DRR image. In the present embodiment, the virtualcamera has been aligned in advance (through the initialization alignmentand/or other one or more adjustment means of facilitatingtwo-dimensional to three-dimensional registration), but, however, thevirtual camera in other embodiment of the present invention hasn't beenadjusted through the initialization alignment and/or other cine or moreadjustment means.

Referring to both FIG. 2 and FIG. 10 operation, the step 3200 isperformed by the processor 1120 to calculate a reference similarityvalue according to the first reconstructed image and the at least onetwo-dimensional image data set. For example, the processor 1120 of thecalculating device 1100 may calculate the reference similarity valueaccording to the first DRR image and the 2D image data.

Reference is now made to both FIG. 2 and FIG. 10 . In operation, thestep 3300 is performed by the processor 1120 to generate a secondreconstructed image from the three-dimensional image data set with asecond spatial parameter. For example, the processor 1120 of thecalculating device 1100 may access the database 1200 and simulate thevirtual camera corresponding to the 3D image data set with the secondspatial parameter so as to generate a second DRR image.

Referring to both FIG. 2 and FIG. 10 , in operation, the step 3400 isperformed by the processor 1120 to calculate a comparison similarityvalue according to the second reconstructed image and the at least onetwo-dimensional image data set. For example, the processor 1120 of thecalculating device 1100 may calculate the comparison similarity valueaccording to the second DRR image and the 2D image data.

Reference is now made to both FIG. 2 and FIG. 10 . In operation, thestep 3500 is performed by the processor 1120 to compare the comparisonsimilarity value with the reference similarity value. For example, theprocessor 1120 of the calculating device 1100 may compare the comparisonsimilarity value with the reference similarity value.

Referring to both FIG. 2 and FIG. 10 , operation, the step 3600 isperformed by the processor 1120 to register the at least one thetwo-dimensional image data set to the three-dimensional image data setif the comparison similarity value is not greater than the referencesimilarity value for computer-assisted surgical navigation based on thetwo-dimensional image data set and the three-dimensional image data setafter registering. For example, if the comparison similarity value isnot greater than the reference similarity value, it means that thesecond DRR image is aligned with the 2D image data. Therefore, theprocessor 1120 of the calculating device 1100 may register the 2D imagedata set from the imaging device 1300 to the 3D image data set stored inthe database 1200. Thereafter, since the 20 image data set from theimaging device 1300 is registered to the 3D image data set stored in thedatabase 1200, the method 3000 in FIG. 10 of the present disclosure canbe used for computer-assisted surgical navigation based on thetwo-dimensional image data set and the three-dimensional image data set.

It is noted that, the present disclosure is not limited to theoperations as shown in FIG. 10 , and it is merely an example forillustrating one of the implements of the present disclosure. Inaddition, the method 3000 in FIG. 10 can be performed by the processor1120 to register the two-dimensional image data set including an APtwo-dimensional image and a LA two-dimensional image to thethree-dimensional image data set from which an AP DRR image and a LA DRRimage derived.

In some embodiments, the steps of generating the first reconstructedimage or the second reconstructed image from the three-dimensional imagedata set with the first reconstructed image or the second reconstructedimage which are performed by the processor comprise: positioning avirtual camera corresponding to a three-dimensional subject composed inaccordance with the first reconstructed image or the secondreconstructed image so as to capture the first reconstructed image orthe second reconstructed image of the three-dimensional subject by thevirtual camera.

For example, the three-dimensional image data set may be a volume ofvoxels obtained after the CT scanning of the object 9000 including thebody of interest. The volume of voxels can be presented in a way ofvolume rendering and then a three-dimensional subject, also known as athree-dimensional model, of the body of interest can be shown on adisplay and stored in the database 1200 simultaneously. The processor1120 may access the database 1200 for the three-dimensional model, andalso determining the position and orientation of a virtual cameracorresponding to the three-dimensional model with the spatial parametergiven to the virtual camera. Once the virtual camera is set. theprocessor 1100 is able to acquire one or a plurality of two-dimensionalimages as the first DRR image or the second DRR image by the DRRalgorithm in the art.

In some embodiments, the virtual camera is defined by a modularizedfunction. The modularized function includes an algorithm and anequation. In some embodiments, the virtual camera is simulated by butnot limited to a ray projection volume rendering.

In the initialization alignment, the virtual camera will be adjusted topreset an initial spatial parameter including a position and/or anorientation of the virtual camera so as to improve the efficiency andenhance the precision of the navigation system 1000. However, formedical purpose there are still insufficiency, and therefore, theadjusted virtual camera after performing the initialization alignmentshould be further refined in the two-dimensional to three-dimensionalregistration for becoming more accurate. In some embodiments, each ofthe first spatial parameter and the second spatial parameter is used todefine a position and/or an orientation of the adjusted virtual cameracorresponding to the three-dimensional subject or the three-dimensionalsubject corresponding to the adjusted virtual camera. As mentionedabove, the adjusted virtual camera should be further refined, and theway to refine the adjusted virtual camera is to adjust spatial parameterincluding a position and/or an orientation of the adjusted virtualcamera. For example, each of the first spatial parameter and the secondspatial parameter is used to define a position and/or an orientation ofthe adjusted virtual camera corresponding to the three-dimensionalsubject or the three-dimensional subject corresponding to the adjustedvirtual camera.

In some embodiments, each of the first spatial parameter and the secondspatial parameter individually comprises one of a position, anorientation, and a parameter comprising the position and theorientation. For example, the first spatial parameter can be a position,an orientation, or a parameter including the position and theorientation, and the second spatial parameter can be a position, anorientation, and a parameter including the position and the orientation.

In some embodiments. each of the comparison similarity value and thereference similarity value is calculated by but not limited to localnormalized correlation (LNC), sum of squared differences (SSD),normalized cross-correlation (NCC), or correlation ratio (CR).

In some embodiments, the three-dimensional image data set (e.g., the 3Dimage data set stored in the database 1200 in FIG. 1 ) is generated byone of a magnetic resonance imaging (MRI) device, an iso-centric imagingfluoroscopic imaging device, an O arm device, a bi-plane fluoroscopydevice, a computed tomography (CT) device, a multi-slice computedtomography (MSCT) device, a high frequency ultrasound (HIFU) device, anoptical coherence tomography (OCT) device, an intra-vascular ultrasound(IVUS) device, a 3D or 4D ultrasound device, and an intraoperative CTdevice.

In some embodiments, the two-dimensional image data set is generated byone of a C-arm fluoroscopic imaging device, a magnetic resonance imaging(MRI) device, an iso-centric C-arm fluoroscopic imaging device, an O-armdevice, a hi-plane fluoroscopy device, a computed tomography (CT)device, a multi-slice computed tomography (MCT) device, a high frequencyultrasound (HIFU) device, an optical coherence tomography (OCT) device,an intra-vascular ultrasound (NUS) device, a two-dimensional,three-dimensional or four-dimensional ultrasound device, and anintraoperative CT device.

In some embodiments, the first spatial parameter is generated inaccordance with a spatial relationship between at least one maker on atwo-dimensional image capturing device and the body of interest. Asmentioned above, the virtual camera will be adjusted to preset theinitial spatial parameter in the initialization alignment so as toimprove the efficiency and enhance the precision of the navigationsystem 1000. and the adjusted spatial parameter after the initializationalignment is subsequently used as the first spatial parameter in thistwo-dimensional to three-dimensional registration. For example, thefirst spatial parameter is generated in accordance with a spatialrelationship between the calibrator 1510 on the X-ray receiver 1320 andthe object 9000 in FIG. 1 .

In some embodiments, the first spatial parameter is a spatial parameterwhich has a maximum similarity among a plurality of similaritiesgenerated in a previous comparison step. For example, the comparisonsteps may be performed many times and each comparison steps willgenerate a similarity. The first spatial parameter is obtained byfinding the parameter that has the maximum similarity among theplurality of similarities.

In some embodiments, the adjusted virtual camera in the beginning of thetwo-dimensional to three-dimensional registration has the first spatialparameter including the first distance and/or the first orientation.When the two-dimensional to three-dimensional registration starts, theadjusted virtual camera will be moved from the first spatial parameterto the second spatial parameter including the second distance and/or thesecond orientation. Therefore, the second spatial parameter is differentfrom the first spatial parameter aspect of defining a distance and/or anorientation of the corresponding virtual camera corresponding to thethree-dimensional image data set (e.g., the three-dimensional image dataset stored in the database 1200 in FIG. 1 ). In more detail, since theadjusted virtual camera described in the present embodiment is amodularized function including an algorithm, equation and one or aplurality of parameters involved therein, the processor 1100 can use theparameters and then stimulate the adjusted virtual cameras at particularlocations in the coordinate system of the three-dimensional image dataset.

In some embodiments, the processor 1120 is further used to perform thefollowing steps: generating a third reconstructed image from thethree-dimensional image data set with a third spatial parameter if thecomparison similarity value is greater than the reference similarityvalue; calculating a second comparison similarity value according to thethird reconstructed image and the at least one two-dimensional imagedata set; comparing the second comparison similarity value with thereference similarity value; and registering the at least one thetwo-dimensional image data set to the three-dimensional image data setif the second comparison similarity value is not greater than thereference similarity value.

For example, if the comparison similarity value is greater than thereference similarity in the previous comparison, the processor 1120 ofthe calculating device 1100 may access the database 1200 and simulatethe adjusted virtual camera corresponding to the 3D image data set at athird location so as, to generate a third DRR image.

Besides, the processor 1120 of the calculating device 1100 may calculatethe second comparison similarity value according to the third AP and/orLA DRR images and the corresponding AP and/or LA two-dimensional imagesof the two-dimensional image data set.

In addition, the processor 1120 of the calculating device 1100 maycompare the second comparison similarity value with the referencesimilarity value.

Furthermore, if the second comparison similarity value is not greaterthan the reference similarity value, it means that the third AP and/orLA DRR images are aligned with the corresponding AP and/or LAtwo-dimensional images of the two-dimensional image data set. Then, thetwo-dimensional image data set acquired by the imaging device 1300 inreal-time operation can be considered being registered to thethree-dimensional image data set acquired previously and pre-stored inthe database 1200.

FIG. 11 depicts a flow diagram of a method 4000 for registering atwo-dimensional image data set of a body of interest with athree-dimensional image data set of the body of interest according toone embodiment of the present disclosure. Reference is now made to bothFIG. 2 and FIG. 11 . In operation, the step 4100 is performed by theprocessor 1120 to simulatively move the adjusted virtual camera from anoriginal or a former spatial position of the coordinate system of thethree-dimensional image data set to a first spatial position, and thestep 4150 is performed by the processor 1120 to generate a first DRRimage, also known as a first reconstructed image, corresponding to thefirst spatial position of the first virtual camera. The step 4200 isthen performed by the processor 1120 to calculate a first similarityvalue according to the first. DRR image and a two-dimensional image,which can be AP or LA two-dimensional image, generated from the imagingdevice 1300, and step 4250 is performed by the processor 1120 to movethe adjusted virtual camera back to the original spatial position.

Subsequently, the step 4300 is performed by the processor 1120 todetermine whether the adjusted virtual camera is moved from the originalspatial position to every spatial position. For example, the adjustedvirtual camera can be moved from the original spatial position to afirst spatial position (e.g., moved from (0,0) to (1,0) in the Cartesiancoordinate system), then moved from the original spatial position to asecond spatial position (e.g., moved from (0,0) to (−1,0) in theCartesian coordinate system), and so on. The movement of the adjustedvirtual camera is preset according to actual requirements. If it isdetermined that the adjusted virtual camera is not moved to everyspatial position, the method 4000 is back to the step 4100 so as to movethe adjusted virtual camera to another spatial position such as a secondspatial position. The steps 4150, 4200 are then performed by theprocessor 1120 to generate another DRR image such as a second DRR image,and calculate another similarity value such as a second similarityvalue. Thereafter, the step 4250 is performed by the processor 1120 tomove the adjusted virtual camera back to the original spatial position.

If it is determined that the adjusted virtual camera is moved to everyspatial position, the method 4000 proceeds to the step 4350. The step4350 is then performed by the processor 1120 to determine whether asimilarity value corresponding to the moved virtual camera is greater.If it is determined that similarity value corresponding to the movedvirtual camera is greater, it means that the DRR image generated by theadjusted virtual camera at the latter position is more similar to thetwo-dimensional image than the DRR image generated by the adjustedvirtual camera at the original or former spatial position. Therefore,the step 4400 is then performed by the processor 1120 to adjust theadjusted virtual camera from the original or former spatial position tothe latter spatial position. The original spatial position described inthe present embodiment represents a position at which the adjustedvirtual camera is located by the processor 1120 in the very beginning,and additionally the former spatial position represents at which theadjusted virtual camera is relocated after an adjustment which mayresult from any of or any combination of the abovementioned alignment orregistration processes such as the initialization alignment.

After the step 4400 is performed, the method 4000 is back to step 4100.The steps 4100, 4150, 4200, 4250, 4300, and 4350 are then performed bythe processor 1120. Referring to the step 4350, if it is determined thatthe similarity value corresponding to the adjusted virtual camera at thelatter position is not greater than any one of the former similarityvalues, the method 4000 proceeds to the step 4450.

After the step 4450 is performed, the step 4500 is performed by theprocessor 1120 to reduce the adjustment. For example, if the previousadjustment is to move the adjusted virtual camera for 1 mm from theformer position in the coordinate system of the three-dimensionalsubject presented by the three-dimensional image data set, the step 4500is performed by the processor 1120 to reduce the adjustment to 0.5 mm.Thereafter, the step 4550 is performed by the processor 1120 todetermine whether the adjustment is less than a preset spatial value,which is for example 0.75 mm. If it is determined that the adjustment isnot less than the preset spatial value, the method 4000 is back to thestep 4100.

If it is determined that the adjustment is less than the preset spatialvalue, it means that the DRR image is adequately aligned with thetwo-dimensional image. Therefore, it can define that the two-dimensionalimage data set from the imaging device 1300 has been registered to thethree-dimensional image data set pre-stored in the database 1200.

In some embodiments, if a difference between the first spatial parameterand the second spatial parameter is not greater than a preset spatialvalue, registering the two-dimensional image data set to thethree-dimensional image data set. For example, as can be seen in thestep 4550 in FIG. 11 , the difference between the first spatial positionand the second spatial position is the adjustment in the step 4550. Ifthe adjustment between the first spatial position and the second spatialposition is not greater than the preset spatial value, it means that theDRR image is aligned with the two-dimensional image data. For example,the present spatial value can be 0.01 mm. If the adjustment is notgreater than 0.01 mm. it means that the DRR image is already alignedwith the two-dimensional image data. Therefore, the processor 1120 ofthe calculating device 1100 may register the 2D image data set from theimaging device 1300 to the 3D image data set stored in the database1200.

It is noted that, the present disclosure is not limited to theoperations as shown ire FIG. 11 , and it is merely an example forillustrating one of the implements of the present disclosure.

FIG. 12 depicts a flow diagram of a method 5000 for registering atwo-dimensional image data set with a three-dimensional image data setof a body of interest according to one embodiment of the presentdisclosure. Reference is now made to both FIG. 2 and FIG. 12 . Inoperation, the step 5100 is performed by the processor 1120 to executean initialization alignment process to the navigation platform 1000 soas to enhance the precision of the navigation platform 1000.

The step 5200 is performed by the processor 1120 to execute a XYlocation alignment process to preset the XY location of the adjustedvirtual camera corresponding to the three-dimensional image data setpre-stored in the database 1200 of the navigation platform 1000.

The step 5300 is performed by the processor 1120 to execute a contracteddrawing alignment to the adjusted virtual camera corresponding to thethree-dimensional image data set pre-stored in the database 1200 of thenavigation platform 1000 preliminarily.

The step 5400 is performed by the processor 1120 to execute an originaldrawing alignment to the adjusted virtual camera corresponding to thethree-dimensional image data set pre-stored in the database 1200 of thenavigation platform 1000.

It is noted that a flow diagram of the XY coordinate alignment in thestep 5200, the thumbnail alignment in the step 5300, and the originalalignment in the step 5400 is shown in FIG. 11 . The difference is thatthe XV coordinate alignment is focused on alignment of the images on theX coordinate and the Y, the thumbnail alignment uses preview or zoom-outimages to achieve a quick alignment, and the original alignment isperformed to further improve the alignment after the preview alignment.

The step 5500 is performed by the processor 1120 to determine whetherrealignment is required. The realignment process is required if an erroror alignment failure determined by the processor 1120 or a judgment froma surgeon. For example, while the difference is always greater than apreset value after several calculation cycles, and the method 5000proceeds to the step 5600 for realignment. On the contrary, the method5000 proceeds to the step 5700 if the alignment is adequate and norealignment process is needed. In that situation, the whole alignmentprocess in FIG. 12 is completed.

In some embodiments, the navigation system 1000 uses low-resolutionimages for preliminary alignment with the two-dimensional images beforestimulation adjusted virtual cameras for generating the first and thesecond reconstructed images. The low-resolution format used herein is toreduce interference or noise signal between two similar images andtherefore facilitate the preliminary alignment. As in the step 5300 ofthe method 5000, the DRR images used in the thumbnail alignment arelow-resolution images.

Realignment

It is noted that, the present disclosure is not limited to theoperations as shown in FIG. 12 , and it is merely an example forillustrating one of the implements of the present disclosure.

The step 5600 for realignment in the method 5000 will be described inthe following FIG. 13 . FIG. 13 depicts a flow diagram of a method 6000for realignment according to one embodiment of the present disclosure.In clinical practice, the two-dimensional image to three-dimensionalregistration procedure fails in the alignment stages from time to time.It may result from neither of AP nor LA two-dimensional images don'tsuccessfully align with the DRR images generated according to thethree-dimensional image data set. However, the alignment failure iscaused more often by only one of the two-dimensional images fails toalign with the corresponding DRR image. Navigation system in prior artis programmed to stop proceed the registration procedure once any kindof failure abovementioned occurs. The consequence is to technicians orsurgeons have to go back to the first step and redo it again. Thissignificantly slows down the whole process of registration. In thepresent embodiment, the navigation system 1000 can address this issue byperforming the realignment stage.

Referring to FIG. 2 and FIG. 13 , if a LA DRR image, which is also knownas LA three-dimensional reconstructed image, doesn't align with the LAC-arm X-ray image, which is also known as the two-dimensional image, andthey need realignment, the processor 1120 may obtain a spatial parameterof a AP virtual camera corresponding to the AP DRR image which issuccessfully registered with the AP C-arm X-ray image in the previoussteps. In view of the above, the AP virtual camera is regard as aregistered virtual camera in the following description, and the LAvirtual camera is regard as an unregistered virtual camera in thefollowing description. However, the present disclosure is not limited tothe above-mentioned embodiments, the AP virtual camera can be theunregistered virtual camera, and the LA virtual camera can be theregistered virtual camera depending on actual situations.

For facilitating the understanding of the method 6000 in FIG. 1reference is made to FIG. 14 , which depicts a schematic diagram of avirtual camera VC according to one embodiment of the present disclosure.It is noted that the virtual camera VC in FIG. 14 is used to illustratethe concept of the present disclosure. The spatial parameter of thevirtual camera VC may include kinds of position and/or orientationinformation such as but not limited to vectors. Both of the registeredvirtual camera and the unregistered virtual camera can be illustrated byFIG. 14 , for example, each of the spatial parameter of the registeredvirtual camera and the unregistered virtual camera may include a vectorof axis Z and an axis Y and also a focal point FP, respectively, definedin the coordination system of the three-dimensional subject generated bythe three-dimensional image data set. Alternatively, the focal point FPmay be accessed by the processor 1120 from the database 1200 separatelyinstead of included in the spatial parameter. For identificationpurpose, the registered virtual camera will be labeled as VC1, and itsaxes and focal point will be labeled as Z1, Y1 and FP1 in the followingdescription. Besides, the unregistered virtual camera will be labeled asVC2, and its axes and focal point will be labeled as Z2, Y2 and FP2 inthe following description.

Referring to FIG. 2 , FIG. 13 , and FIG. 14 , in operation, the step6100 is performed by the processor 1120 to obtain a first vector from aspatial parameter of a registered virtual camera in the coordinatesystem of the three-dimensional image data set. For example, theprocessor 1120 may obtain the axis Z1 from the spatial parameter of theregistered virtual camera VC1 in a coordinate system of thethree-dimensional image data set pre-stored in the database 1200.

The step 6200 is performed by the processor 1120 to obtain a firsttransformed vector from a spatial parameter of an unregistered virtualcamera in the coordinate system of the three-dimensional image data setby transforming the first vector of the registered virtual camerathrough at least one transforming matrix. For example, the processor1120 may obtain the axis Z2 from the spatial parameter of theunregistered virtual camera VC2 in the coordinate system of thethree-dimensional image data set by transforming the axis Z1 of theregistered virtual camera VC1 through a programmed or pre-storedtransforming matrix which is established by the processor 1120 accordingto the spatial relation of the particular components in the navigationsystem 1000.

The step 6300 is performed by the processor 1120 to obtain a focal pointof the unregistered virtual camera at a reference point of theunregistered LA X-ray image, which is the unregistered two-dimensionalimage of the two-dimensional image data set, in the coordinate system ofthe three-dimensional image data set. For example, the processor 1120may obtain a focal point FP2 of the unregistered virtual camera VC2 at acentral point of calibrators of the two-dimensional image in thecoordinate system of the three-dimensional image data set according tothe previously performed unsuccessful registration.

The step 6400 is performed by the processor 1120 to reposition theunregistered virtual camera according to the first transformed vectorand the focal point of the unregistered virtual camera for generating anupdate reconstructed image based on reposition of the unregisteredvirtual camera. For example, the unregistered virtual camera VC2 may besimulatively moved from its original position to the position calculatedby the processor 1120 according to the axis Z2 of the unregisteredvirtual camera VC2 and the focal point FP2 of the unregistered virtualcamera VC2.

In some embodiments, the first vector of the registered virtual camerais from the position of the registered virtual camera to the focal pointof the registered virtual camera. For example, as shown in FIG. 14 , theaxis Z1 of the registered virtual camera VC1 is from the position of theregistered virtual camera VC1 to the focal point FP1 of the registeredvirtual camera VC1.

In some embodiments, the first vector is defined as the Z axis (e.g.,the axis Z1 in FIG. 14 ) of the registered virtual camera (e.g., theregistered virtual camera VC1 in FIG. 14 ) in a coordinate system of thethree-dimensional image data set.

In some embodiments, the processor 1120 is configured to obtain theplurality of commands from the memory 1110 to perform following steps:obtaining a second vector from a spatial parameter of the registeredvirtual camera in the coordinate system of the three-dimensional imagedata set; and obtaining a second transformed vector from a spatialparameter of the unregistered virtual camera in the coordinate system ofthe three-dimensional image data set according to the second vector ofthe registered virtual camera

For example, the processor 1120 may obtain the axis Y1 from the spatialparameter of the registered virtual camera VC1 in a coordinate system ofthe three-dimensional image data set. The processor 1120 may obtain theaxis Y2 three-dimensional of the unregistered virtual camera VC2 in thecoordinate system of the three-dimensional image data set according tothe axis Y1 from the spatial parameter of the registered virtual cameraVC1.

In some embodiments, the second vector is from the central point (e.g.,the point FP in FIG. 14 ) to the top edge (e.g., the top edge in FIG. 14) of a reconstructed image obtained by the registered virtual camera(e.g., the registered virtual camera VC1 in FIG. 14 ) parallel to thereconstructed image.

In some embodiments, the second vector is defined as the Y axis (e.g.,the axis Y1 in FIG. 14 ) from the spatial parameter of the registeredvirtual camera (e.g., the registered virtual camera VC1 in FIG. 14 ) ina coordinate system of the three-dimensional image data set.

It is noted that, the present disclosure is not limited to thestructures and the operations as shown in FIG. 13 and FIG. 14 , and itis merely an example for illustrating one of the implements of thepresent disclosure,

In some embodiments, the two-dimensional image data set comprises afirst and second two-dimensional images, and the at least onetransforming matrix similar or identical to the previously disclosedherein comprises a first matrix which functions to transform thecoordinate system of the first two-dimensional image to the coordinatesystem of the second two-dimensional image. For example, thetwo-dimensional image data set includes an AP two-dimensional image anda LA two-dimensional image, and the transforming matrix includes a firstmatrix which functions to transform the coordinate system of the APtwo-dimensional image to the coordinate system of the LA two-dimensionalimage.

In some embodiments, the reference point is at the central point of acalibrator module in the two-dimensional image data set in thecoordinate system of the three-dimensional image data set. For example,the reference point is at the central point, which is also called theorigin, of a calibrator module which is shown on the unregistered AP orLA X-ray two-dimensional image included in the two-dimensional imagedata set. The reference point is simulatively positioned and orientatedin the coordinate system of the three-dimensional image data set forfurther function by the program of the navigation system 1000.

In some embodiments, the at least one transforming matrix comprisessecond matrix and a third matrix, wherein the second matrix whichfunctions to transform the coordinate system of a reference mark to thecoordinate system of the three-dimensional image data set and the thirdmatrix which functions to transform the coordinate system of thereference mark to the coordinate system of a tracker module. Forexample, the at least one transforming matrix includes a second matrixand a third matrix. The second matrix which functions to transform thecoordinate system of the calibrators 1510, 1530 and the dynamicreference frames 1520A in FIG. 1 to the coordinate system of the 3Dimage data set store in the database 1200 in FIG. 1 . The third matrixwhich functions to transform the coordinate system of the calibrators1510, 1530 and the dynamic reference frames 1520A in FIG. 1 to thecoordinate system of the tracker 1400 in FIG. 1 . The transformingmatrix and the transformation of coordinate systems are well-known inthe state of art. More, information can be found in L Dorst, etc.,Geometric Algebra For Computer Science, published by Morgan KaufmannPublishers and M. N. Oosterom, etc., Navigation of a robot-integratedfluorescence laparoscope in preoperative SPECT/CT and intraoperativefreehand SPECT imaging data: A phantom study, August 2016, Journal ofBiomedical Optics 21(8):086008, which are incorporated herein byreference.

The step 5600 for realignment in the method 5000 in FIG. 12 will bedescribed in the following FIG. 15 . FIG. 15 depicts a flow diagram of amethod 7000 for realignment according to one embodiment of the presentdisclosure. Referring to FIG. 2 and FIG. 15 , if an unregistered virtualcamera corresponding to the vertebra V1 as shown in FIG. 5 or FIG. 7needs realignment, the processor 1120 may find a registered virtualcamera corresponding to the vertebra V0 or V2 as shown in FIG. 5 or FIG.7 .

Referring to FIG. 2 and FIG. 15 , in operation, the step 7100 isperformed by the processor 1120 to obtain a first spatial parameter of afirst registered virtual camera, wherein the first registered virtualcamera is positioned corresponding to a first two-dimensional image ofthe two-dimensional image data set. For example, the processor 1120 mayobtain the first spatial parameter of the registered virtual cameracorresponding to the vertebra V0 or V2 as shown in FIG. 5 or FIG. 7 .The registered virtual camera is positioned corresponding to thetwo-dimensional image related to the vertebra V0 or V2 as shown in FIG.4 or FIG. 6 .

Referring to FIG. 2 and FIG. 15 , in operation, the step 7200 isperformed by the processor 1120 to adjust a second spatial parameter ofthe first unregistered virtual camera with the first spatial parameterof the first registered virtual camera for generating an updatereconstructed image based on reposition of the first unregisteredvirtual camera, wherein the first unregistered virtual camera is failedto be positioned corresponding to the first two-dimensional image of thetwo-dimensional image data set. For example, the processor 1120 mayadjust the second spatial parameter of the unregistered virtual cameracorresponding to the vertebra V1 as shown in FIG. 6 or FIG. 7 with thefirst spatial parameter of the registered virtual camera correspondingto the vertebra V0 or V2 as shown in FIG. 5 or FIG. 7 . The unregisteredvirtual camera is failed to be positioned corresponding to thetwo-dimensional image related to the vertebra V1 as shown in FIG. 4 orFIG. 6 .

In some embodiments, a first part of the body of interest is included inthe first two-dimensional image, a second part of the body of interestis included in the first two-dimensional image, and the first part andthe second part of the body of interest are adjacent. For example, theobject 9000 is a patient under vertebral disorder which needs anoperation for stabilizing three vertebral levels. The three vertebrallevels are together considered as the body of interest and shown in FIG.4 and FIG. 6 . According to the success or failure of the alignment withthe three-dimensional image data set, the vertebral levels can becategorized into two parts.

The first part includes the vertebra V0 or V2, and the second partincludes the vertebra V1. In more detail, the first part of the body ofinterest in FIG. 1 is included in the first two-dimensional imagescorresponding to the vertebra V0 or V2. The first two-dimensional imagesare FIG. 4 as the AP X-ray image and FIG. 6 as the LA X-ray image. Thesecond part of the body of interest in FIG. 1 is included in the firsttwo-dimensional images corresponding to the vertebra V1. The firsttwo-dimensional images are FIG. 4 and FIG. 6 . As can be seen in FIG. 4and FIG. 6 , the vertebra V0 or V2 included in the first part of theobject 9000 is adjacent to the vertebra V1 included in the second partof the object 9000 corresponding to the vertebra V1.

In some embodiments, the first part of the body of interest is definedaccording to a first marker in the first two-dimensional image of thetwo-dimensional image data set, and the second part of the body ofinterest is defined according to a second marker in the firsttwo-dimensional image of the two-dimensional image data set. Forexample. the first part of the body of interest of the object 9000 inFIG. 1 is defined by the program of the navigation system 1000 toautomatically recognize the image boundary of the vertebra which is markas V0 or V2 on the two-dimensional images in FIG. 4 and FIG. 6 . Thesecond part of the body of interest of the object 9000 in FIG. 1 isdefined by the program of the navigation system 1000 to automaticallyrecognize the image boundary of the vertebra which is mark as V1 on thetwo-dimensional image in FIG. 4 and FIG. 6 by the same measure as thefirst part.

In some embodiments, the first spatial parameter of the first registeredvirtual camera comprises a position and/or an orientation data which areused to define a position and/or an orientation of the first registeredvirtual camera corresponding to the three-dimensional subject or thethree-dimensional subject corresponding to the first registered virtualcamera. For example, the first spatial parameter of the registeredvirtual camera corresponding to the vertebra V0 or V2 as shown in FIG. 5or FIG. 7 includes a position and/or an orientation data which are usedto define a position and/or an orientation of the registered virtualcamera corresponding to the three-dimensional object generated from thethree-dimensional image data set or the three-dimensional objectcorresponding to the registered virtual camera.

In some embodiments, the second spatial parameter of the firstunregistered virtual camera comprises a position and/or an orientationdata which are used to define a position and/or an orientation of thefirst unregistered virtual camera corresponding to the three-dimensionalsubject or the three-dimensional subject corresponding to the firstregistered virtual camera. For example, the second spatial parameter ofthe unregistered virtual camera corresponding to the vertebra V1 asshown in FIG. 5 or FIG. 7 includes a position and/or an orientation datawhich are used to define a position and/or an orientation of theunregistered virtual camera corresponding to the three-dimensionalobject generated from the three-dimensional image data set or thethree-dimensional object corresponding to the registered virtual camera.

In some embodiments, the first registered virtual camera is positionedaccording to comparison of the similarity values calculated according todifferent reconstructed images obtained from the three-dimensional imagedata set and the first two-dimensional image of the two-dimensionalimage data. For example, the registered virtual camera is positionedaccording to comparison of the similarity values calculated according todifferent CARR images obtained from the three-dimensional image data setand the two-dimensional image related to the vertebra V0 or V2 as shownin FIG. 4 or FIG. 6 .

In some embodiments, each of the similarity values is calculated bylocal normalized correlation (LNC), sum of squared differences (SSD),normalized cross-correlation (NCC), or correlation ratio (CR).

in some embodiments, the processor 1120 is further used to perform thefollowing steps: defining the second spatial parameter of the firstunregistered virtual camera to be a N spatial parameter; determiningwhether a N-M spatial parameter of the first registered virtual camerais positioned corresponding to the first two-dimensional image of thetwo-dimensional image data set, wherein N and M are integers, and M isless than N, and if the N-M spatial parameter of the first registeredvirtual camera is positioned corresponding to the first two-dimensionalimage of the two-dimensional image data set, defining the N-M spatialparameter to be the first spatial parameter of the first registeredvirtual camera.

For example. the processor 1120 of the calculating device 1100 maydefine the second spatial parameter of the unregistered virtual camerato be a N spatial parameter. The processor 1120 of the calculatingdevice 1100 may determine whether a N-M spatial parameter of theregistered virtual camera is positioned corresponding to thetwo-dimensional image related to the vertebra V0 or V2 as shown in FIG.4 or FIG. 6 , wherein N and M are integers, and M is less than N. Theprocessor 1120 of the calculating device 1100 may define the N-M spatialparameter to be the first spatial parameter of the registered virtualcamera if the N-M spatial parameter of the registered virtual camera ispositioned corresponding to the two-dimensional image related to thevertebra V0 or V2 as shown in FIG. 4 or FIG. 6 . Specifically, the Nspatial parameter is corresponding to the unregistered virtual camera.The navigation system 1000 may find the N-M spatial parameter which iscorresponding to the registered virtual camera, and the N-M spatialparameter will be used in the unregistered virtual camera forfacilitating the process.

In some embodiments, the processor 1120 is further used to perform thefollowing steps: defining the second spatial parameter of the firstunregistered virtual camera to be a N spatial parameter; determiningwhether a N+M spatial parameter of the first registered virtual camerais positioned corresponding to the first two-dimensional image of thetwo-dimensional image data set, wherein N and M are integers, and M isless than N; and if the spatial parameter of the first registeredvirtual camera is positioned corresponding to the first two-dimensionalimage of the two-dimensional image data set, defining the N+M spatialparameter to be the first spatial parameter of the first registeredvirtual camera.

For example, the processor 1120 of the calculating device 1100 maydefining the second spatial parameter of the unregistered virtual camerato be a N spatial parameter. The processor 1120 of the calculatingdevice 1100 may determine whether a N+M spatial parameter of theregistered virtual camera is positioned corresponding to thetwo-dimensional image related to the vertebra V0 or V2 as shown in FIG.4 or FIG. 6 wherein N and M are integers, and M is less than N. Theprocessor 1120 of the calculating device 1100 may define the N+M spatialparameter to be the first spatial parameter of the registered virtualcamera if the N+M spatial parameter of the first registered virtualcamera is positioned corresponding to the two-dimensional image relatedto the vertebra V0 or V2 as shown in FIG. 4 or FIG. 6 .

It is noted that, the present disclosure is not limited to theoperations as shown in FIG. 15 , and it is merely an example forillustrating one of the implements of the present disclosure.

As discussed above, the step 5600 for realignment in the method 5000 inFIG. 12 includes but not limited to two realignment processes in FIG. 13and FIG. 15 . For facilitating the understanding regarding therealignment processes in FIG. 13 and FIG. 15 , reference is made to FIG.16 , which depicts a schematic diagram of a flow diagram of a method8000 for realignment according to one embodiment of the presentdisclosure.

Reference is now made to both FIG. 2 and FIG. 16 . In operation, thestep 8200 is performed by the processor 1120 to determine whether one ofthe AP DRR image acquired by the AP virtual camera and the LA DRR imageacquired by the LA virtual camera is not registered. If it is determinedthat one of the DRRs acquired by the AP virtual camera or the LA virtualcamera is not registered, the method 8000 proceeds to step 8300.Specifically, if the AP virtual camera is not registered, the step 8300is performed by the processor 1120 to reset the AP virtual camera whichis not registered according to the LA virtual camera which isregistered. The method 6000 in FIG. 13 is similar to the step 8300 inmethod 8000. When the DRR image acquired by the AP virtual camera or theLA virtual camera is not registered, the method 6000 will find out anduse a registered virtual camera to reset the unregistered virtualcamera. Subsequently, the step 8700 is performed by the processor 1120to execute the original drawing alignment by using the reset virtualcamera, In addition, the step 8400 is performed by the processor 1120 toadjust the ROI region for avoiding the interference,

After the step 8200 is performed, if it is determined that none of theDRR images acquired by the AP virtual c era and the LA virtual camera isregistered, the method 8000 proceeds to step 8500. Specifically, if theAP virtual camera and the LA virtual camera are not registered, it meansthe AP and LA DRR images corresponding to a first vertebra are allfailed to be registered. The step 8500 is performed by the processor1120 to find out another AP virtual camera and another LA virtual cameracorresponding to a second vertebra, wherein the DRRs of the virtualcameras are successfully registered with the three-dimensional imagedata set. Thereafter, the processor 1120 may reset the unregistered APvirtual camera and the unregistered LA virtual camera corresponding tothe first vertebra according to the spatial parameter or data of theregistered AP virtual camera and the registered LA virtual cameracorresponding to the second vertebra. The method 7000 in FIG. 15 issimilar to the step 8500 in method 8000. When the DRR images acquired bythe AP virtual camera and the LA virtual camera corresponding to thefirst vertebra are all failed to be registered, the method 7000 willfind out and use the spatial parameter or data of the registered APvirtual camera and the registered LA virtual camera corresponding to thesecond vertebra to reset the unregistered AP virtual camera and theunregistered LA virtual camera corresponding to the first vertebra.Subsequently, the step 8700 is performed by the processor 1120 toexecute the original drawing alignment by using the reset virtualcamera. In addition, the step 8600 is performed by the processor 1120 toadjust the ROI region for avoiding the interference

It can be understood from the embodiments of the present disclosure thatapplication of the present disclosure has the following advantages. Themethod and the navigation system for registering a two-dimensional imagedata set with a three-dimensional image data set of a body of interestof the present disclosure can pre-store three-dimensional image data setof the body of interest in the database, and then take merely two X-rayimages (two-dimensional images) of the patient (the body of interest)during the surgery so as to establish the relation between thetwo-dimensional image data set and the three-dimensional image data set.Thereafter, the method and the navigation system of the presentdisclosure may provide an accurate navigation during the surgery byusing the pre-store three-dimensional image data set. Since the methodand the navigation system of the present disclosure merely take twoX-ray images (two-dimensional images) of the body of interest, theradiation exposure to the patient (the body of interest) is reduced byover 98%. In view of the above, the present disclosure may provide themethod and the navigation system for executing the two-dimensional tothree-dimensional registration in a more accurate and efficient way.

Although the present invention has been described in considerable detailwith reference to certain embodiments thereof, other embodiments arepossible. Therefore, the spirit and scope of the appended claims shouldnot be limited to the description of the embodiments contained herein.

It will be apparent to those skilled in the art that variousmodifications and variations can be made to the structure of the presentinvention without departing from the scope or spirit of the invention.In view of the foregoing, it is intended that the present inventioncover modifications and variations of this invention provided they failwithin the scope of the following claims and their equivalents.

What is claimed:
 1. A method for registering a two-dimensional imagedata set with a three-dimensional image data set of a body of interest,the method comprising: adjusting a first virtual camera according to adistance parameter calculated corresponding to the two-dimensional imagedata set and the body of interest; rotating the first virtual cameraaccording to an angle difference between a first vector and a secondvector, wherein the first vector is calculated from two spatial marks inthe three-dimensional image data set, and the second vector iscalculated from two first plain marks in the two-dimensional image dataset; and rotating the first virtual camera according to an angle whichis corresponding to a maximum similarity value of a plurality ofsimilarity values calculated in accordance with reconstructed images ofthe three-dimensional image data set which includes one generated by thefirst virtual camera and the others generated by other virtual cameraswith different angles or different pixels from the one generated by thefirst virtual camera and the two-dimensional image data set forimplementing in the two-dimensional image data set to thethree-dimensional image data set registration of a navigation systemafter adjusting and rotating.
 2. The method of claim 1, wherein thedistance parameter is calculated according to the two first plain marksin the two-dimensional image data set, and the method comprising:adjusting the reconstructed images generated by the first virtual cameraaccording to the distance parameter.
 3. The method of claim 1, whereinthe distance parameter is calculated according to a distance between anestimated position of a emitter and an estimated position of the body ofinterest.
 4. The method of claim 3, wherein the estimated position ofthe emitter is calculated according to the two-dimensional image dataset, and the estimated position of the body of interest is at a positionat which a first virtual line and a second virtual line are the closest.5. The method of claim 4, wherein the two-dimensional image data setincludes first and second two-dimensional images, the first virtual lineis generated between the estimated position of the emitter and a centralpoint of at least two reflectors which are radiated when a firsttwo-dimensional image data set is captured, the second virtual line isgenerated between the estimated position of the emitter and the centralpoint of the at least two reflectors which are radiated when a secondtwo-dimensional image data set is captured.
 6. The method of claim 1,wherein the two-dimensional image data set includes a firsttwo-dimensional image, the first vector is calculated from the twopartial marks in the three-dimensional image data set generated by thefirst virtual camera, and the second vector is calculated by the twofirst plain marks in a first two-dimensional image data.
 7. The methodof claim 6, wherein the two-dimensional image data set includes a secondtwo-dimensional image, the second vector is calculated from two secondplain marks in a second two-dimensional image data.
 8. The method ofclaim
 1. wherein an adjusted first virtual camera is generated after therotation of the first virtual camera, and the method comprise: rotatingthe adjusted first virtual camera according to the angle which iscorresponding to an adjusted maximum similarity value of the pluralityof similarity values calculated in accordance with adjustedreconstructed images which includes one generated by the adjusted firstvirtual camera and the others generated by other virtual cameras withdifferent angles from the one generated by the adjusted first virtualcamera and the two-dimensional image data set.
 9. The method of claim 1,wherein an adjusted maximum similarity value of the plurality ofsimilarity values calculated in accordance with adjusted reconstructedimages which includes one generated by an adjusted first virtual cameraand the others generated by other virtual cameras with different anglesor different pixels from the adjusted one generated by the adjustedfirst virtual camera and the two-dimensional image data set.
 10. Themethod of claim 1, further comprising: adjusting the first virtualcamera of a plurality of virtual cameras corresponding to thethree-dimensional image data set according to a matrix corresponding tothe two-dimensional image data set.
 11. A navigation system forregistering a two-dimensional image data set with a three-dimensionalimage data set of a body of interest, comprising: a memory, configuredto store a plurality of commands; and a processor, configured to obtainthe plurality of commands from the memory to perform following steps:adjusting a first virtual camera according to a distance parametercalculated corresponding to the two-dimensional image data set and thebody of interest rotating the first virtual camera according to an angledifference between a first vector and a second vector, wherein the firstvector is calculated from two spatial marks in the three-dimensionalimage data set, and the second vector is calculated from two first plainmarks in the two-dimensional image data set; and rotating the firstvirtual camera according to an angle which is corresponding to a maximumsimilarity value of a plurality of similarity values calculated inaccordance with reconstructed images of the three-dimensional image dataset which includes one generated by the first virtual camera and theothers generated by other virtual cameras with different angles ordifferent pixels from the one generated by the first virtual camera andthe two-dimensional image data set for implementing in thetwo-dimensional image data set to the three-dimensional image data setregistration of the navigation system after adjusting and rotating. 12.The navigation system of claim 11, wherein the distance parameter iscalculated according to the two first plain marks in the two-dimensionalimage data set, and the processor is further configured to, performfollowing step: adjusting the reconstructed images generated by thefirst virtual camera according to the distance parameter.
 13. Thenavigation system of 11, wherein the distance parameter is calculatedaccording to a distance between an estimated position of an emitter andan estimated position of the body of interest.
 14. The navigation systemof claim 13, wherein the estimated position of the emitter is calculatedaccording to the two-dimensional image data set, and the estimatedposition of the body of interest is at a position at which a firstvirtual line and a second virtual line are the closest,
 15. Thenavigation system of claim 14, wherein the two-dimensional image dataset includes first and second two-dimensional images, the first virtualline is generated between the estimated position of the emitter and acentral point of at least two reflectors which are radiated when a firsttwo-dimensional image data set is captured, the second virtual line isgenerated between the estimated position of the emitter and the centralpoint of the at least two reflectors which are radiated when a secondtwo-dimensional image data set is captured.
 16. The navigation system ofclaim 11, wherein the two-dimensional image data set includes a firsttwo-dimensional image, the first vector is calculated from the twospatial arks in the three-dimensional image data set generated by thefirst virtual camera, and the second vector is calculated by the twofirst plain marks in a first two-dimensional image data.
 17. Thenavigation system of claim 16, wherein the two-dimensional image dataset includes a second two-dimensional image, the second vectorcalculated from two second plain marks in a second two-dimensional imagedata.
 18. The navigation system of claim 11, wherein an adjusted firstvirtual camera is generated after the rotation of the first virtualcamera, and the processor is further configured to perform followingstep: rotating the adjusted first virtual camera according to the anglewhich is corresponding to a adjusted maximum similarity value of theplurality of similarity values calculated in accordance with adjustedreconstructed images which includes one generated by the adjusted firstvirtual camera and the others generated by other virtual cameras withdifferent angles from the one generated by the adjusted first virtualcamera and the two-dimensional image data set.
 19. The navigation systemof claim 11, wherein an adjusted maximum similarity value of theplurality of similarity values calculated in accordance with adjustedreconstructed images which includes one generated by an adjusted firstvirtual camera and the others generated by other virtual cameras withdifferent angles or different pixels from the adjusted one generated bythe adjusted first virtual camera and the two-dimensional image dataset.
 20. The navigation system of claim 11, wherein the processor isfurther configured to perform following step: adjusting the firstvirtual camera of a plurality of virtual cameras corresponding to thethree-dimensional image data set according to a matrix corresponding tothe two-dimensional image data set.